Skip to content

fix: use structured YAML parsing for frontmatter and scripts in process_template - #4552

Open
darion-yaphet wants to merge 1 commit into
github:mainfrom
darion-yaphet:fix/structured-yaml-parsing
Open

fix: use structured YAML parsing for frontmatter and scripts in process_template#4552
darion-yaphet wants to merge 1 commit into
github:mainfrom
darion-yaphet:fix/structured-yaml-parsing

Conversation

@darion-yaphet

@darion-yaphet darion-yaphet commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Description

In `IntegrationBase.process_template`, extracting and stripping the `scripts:` block from command templates previously relied on a hand-rolled line-by-line

scanning loop and regex matching (script_pattern = re.compile(r"^\s*([A-Za-z0-9_-]+):\s*(.+)$")).

This approach had multiple issues:
- It failed on valid YAML syntax in frontmatter, such as comments between script entries, quoted script paths, or folded/literal multi-line block scalars (`>` or

|).
- It risked accidental matches if scripts: was referenced inside Markdown body text or code examples.
- Stripping scripts: relied on line indentation checks rather than structured object manipulation.

This PR replaces the hand-rolled text traversal with structured YAML handling using `yaml.safe_load` and `yaml.dump`:
1. Safely parses frontmatter into a Python dictionary to extract `scripts`.
2. Replaces `{SCRIPT}` in the template body.
3. Removes `scripts` (`frontmatter_dict.pop("scripts", None)`) and dumps the cleaned frontmatter back via `yaml.dump(..., sort_keys=False, allow_unicode=True,

width=float("inf")). 4. Adds unit tests in tests/integrations/test_base.py` covering YAML comments, quoted script commands, and folded scalar descriptions.

Testing

- [x] Tested locally with `uv run specify --help`
- [x] Ran existing tests with `uv sync && uv run pytest`
  - `pytest tests/integrations/test_base.py tests/test_command_template_py_scripts.py` (118 tests passed)
  - `ruff check src/specify_cli/integrations/base.py tests/integrations/test_base.py` (All checks passed)
- [ ] Tested with a sample project (if applicable)

## AI Disclosure

- [ ] I **did not** use AI assistance for this contribution
- [x] I **did** use AI assistance (describe below)

Assisted by Google Antigravity / Gemini for refactoring `process_template` to use `yaml.safe_load` and `yaml.dump`, adding test coverage in

tests/integrations/test_base.py, and drafting commit/PR descriptions.

…ss_template

Replace hand-rolled line-by-line scanning and regex matching in IntegrationBase.process_template with yaml.safe_load and yaml.dump. Properly support YAML comments, folded block scalars, and quoted values in command templates while stripping the scripts key cleanly. Add unit tests in tests/integrations/test_base.py to verify structured YAML frontmatter parsing.
@mnriem mnriem added the triage-can-wait Verdict: valid and in-scope but deprioritized; held behind the evidence gate label Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-can-wait Verdict: valid and in-scope but deprioritized; held behind the evidence gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants